Refined Pruning Techniques for Feed-forward Neural Networks

Pruning algorit hms for feed-forward neur al networks typically have the undesirable side effect of int erfering with t he learning pro cedure. The network reduct ion algorithm presented in this pap er is implemented by considering only directions in weight space that are orthogonal to those required by t he learning algorit hm. In this way, the network redu ction algorithm chooses a minimal network from amon g the set of networks with constant E-function values. It thus avoids introducing any inconsistency with learning by explicit ly using t he redundancy inherent in an oversize network. The method is tested on boolean problems and shown to be very useful in practice.